The background description provided herein is for the purpose of generally presenting the context of the disclosure. Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.
In software, a stack buffer overflow or stack buffer overrun occurs when a function writes to a memory address on the function's call stack outside of an intended data structure—usually a fixed-length buffer or an allocated portion thereof. These can occur accidentally through stack buffer overflow bugs or though deliberate attacks by malware; in the latter case, it is sometimes referred to as “stack smashing”. If an untrusted function is able to fill some or all the stack buffer with data, then that function can inject data into the stack and take control of functions executed by the central processing unit; the injected data may be data used by a function (such as parameters, return values, and addresses) or may be executable code.
Protection against stack smashing attacks is not effective when malware can bypass code that verifies stack integrity. Checking the stack to verify that the trusted function is entered from another trusted code is not reliable because malware can manipulate the stack, faking a trusted function's return address using return-oriented programming. In return-oriented programming, attacking malware gains control of the call stack, typically via a buffer overrun, to overwrite a function's variables and/or return address. The overwritten return address may then be used to redirect control flow to the (corrupted) calling subroutine to execute carefully chosen machine instruction sequences called “gadgets”. A gadget typically ends in a return instruction and is located in an existing program and/or shared library code. Chained together, gadgets may allow an attacker to perform arbitrary operations. Malware can take advantage of such vulnerabilities by jumping to code branches inside the trusted function after setting the machine state necessary to achieve malicious goals.
Some legacy solutions inject “stack canaries” to detect stack manipulation. A stack canary is typically a small integer with a random value chosen at the start of function execution and placed in the stack before the stack return pointer. Because stack smashing malware sometimes overwrites the return pointer to take control of function execution, the canary value may be overwritten. However, these software-only-solutions cannot detect if the malware manages to modify the stack without a buffer overflow or without overwriting return pointers, thus not modifying the canary words.